Practical Formal Verification of MPI and Thread Programs

Large-scale simulation codes in science and engineering are written using the Message Passing Interface (MPI). Shared memory threads are widely used directly, or to implement higher level programming abstractions. Traditional debugging methods for MPI or thread programs are incapable of providing useful formal guarantees about coverage. They get bogged down in the sheer number of interleavings (schedules), often missing shallow bugs. In this tutorial we will introduce two practical formal verification tools: ISP (for MPI C programs) and Inspect (for Pthread C programs). Unlike other formal verification tools, ISP and Inspect run directly on user source codes (much like a debugger). They pursue only the relevant set of process interleavings, using our own customized Dynamic Partial Order Reduction algorithms. For a given test harness, DPOR allows these tools to guarantee the absence of deadlocks, instrumented MPI object leaks and communication races (using ISP), and shared memory races (using Inspect). ISP and Inspect have been used to verify large pieces of code: in excess of 10,000 lines of MPI/C for ISP in under 5 seconds, and about 5,000 lines of Pthread/C code in a few hours (and much faster with the use of a cluster or by exploiting special cases such as symmetry) for Inspect. We will also demonstrate the Microsoft Visual Studio and Eclipse Parallel Tools Platform integrations of ISP (these will be available on the LiveCD). The attendees of this tutorial will be given a LiveCD containing ISP and Inspect that they can boot into on their laptops (Win or Mac). In the forenoon session, they will be given a sufficient understanding of practical dynamic analysis methods and DPOR methods to practice them on simple examples. In the afternoon session, they will be able to work through larger examples, and also learn in depth the details of ISP and Inspect algorithms. This is joint work with our students Sarvani Vakkalanka, Yu Yang, Anh Vo, Michael DeLisi, Sriram Aananthakrishnan, Subodh Sharma, Simone Atnezi, Greg Szubzda, Jason William, Alan Humphrey, Chris Derrick, Wei-Fan Chiang, Guodong Li, and Geof Sawaya. The work is supported by Microsoft, NSF CNS 0509379 and CCF-0811429, and SRC Task ID 1847.001.